Additional Feedback: - From Table 2, it appears that the generalization of robustness by adversarial training to unseen attacks is boosted by the addition of OM-AT, although still mainly due AT. How are the results in such metric combining instead AT with other kind of perturbations, e.g. Is the improvement due to on-manifold adversarial training or just to more diverse adversarial attacks seen at training time? - Do the authors have an intuition about how the results presented can be useful in practice without the assumption of knowing the exact manifold? As mentioned above, it seems that the benefit of DMAT against unseen attacks decreases with out-of-manifold images. After reading it and the other reviews, I see positively the contribution/experiments on the artificial dataset. In particular, showing the benefit from OM-AT for clean accuracy and OM-robustness also in the domain of natural images is meaningful, combining AT and OM-AT seems also novel, although methodologically quite straightforward and DMAT leads to better robustness against unforeseen attacks (Table 2) on the artificial dataset OM-ImageNet.

Adversarial training is a popular defense strategy against attack threat models with bounded Lp norms. However, it often degrades the model performance on normal images and more importantly, the defense does not generalize well to novel attacks. Given the success of deep generative models such as GANs and VAEs in characterizing the underlying manifold of images, we investigate whether or not the aforementioned deficiencies of adversarial training can be remedied by exploiting the underlying manifold information. To partially answer this question, we consider the scenario when the manifold information of the underlying data is available. We use a subset of ImageNet natural images where an approximate underlying manifold is learned using StyleGAN.